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Abstract — The ability of information processing in biologically 
motivated Boolean networks is of interest in recent information 
theoretic research. One measure to quantify this ability is the well 
known mutual information. Using Fourier analysis we show that 
canalizing functions maximize the mutual information between 
an input variable and the outcome of the function. We proof our 
result for Boolean functions with uniform distributed as well as 
product distributed input variables. 

I. Introduction 

In systems and computational biology Boolean networks 
are widely used to model various dependencies. One major 
application of such networks are regulatory networks H], ||2|- 
In this context Boolean networks in general and consequently 
Boolean functions have been extensively studied in the past. 

One focus of research is the information processing in 
such networks. It has been stated, that networks, whose 
dynamical behavior is critical, i.e. at the edge between stable 
and chaotic behavior, have somehow an optimized information 
processing ability |3 |, (4], (5). Further it has been shown, that 
biologically motivated networks have critical dynamics [l], 
||6l. The dynamics depend on two major characteristics of the 
network, the topology and the choice of functions. Here, so- 
called canalizing Boolean functions seem to have a stabilizing 
effect Q, |I8|. It has been shown 0, U^, that regulatory 
networks, such as the regulatory network of Escherichia Coli, 
consist mainly of canalizing functions. 

One way to formalize and to quantify information proces- 
sion abilities is the well-known mutual information based on 
Shannon's theory ifTTIl . The mutual information is a measure 
for the statistical relation between some input and output of 
a system, as for example shown by Fano's inequality |12|. It 
has akeady been applied in the context of Boolean networks 
and functions, such as cellular automata |3J, random Boolean 
networks |5l, fT3\ and iterated function systems (T^. 

In this paper we use Fourier analysis to investigate Boolean 
functions. This has first been addressed in ifTSll . Recently 
a relation between the mutual information and the Fourier 
spectra has been derived 1 16|. In particular, it has been shown 
that the mutual information between one input variable and 
the output only depends on two coefficients, namely the zero 
coefficient and the first order coefficient of this variable. 
Further a relation between the canalizing property and these 
coefficients has been derived |17|. In this paper we combine 



two approaches to show that canalizing functions maximize 
the mutual information for a given expectation value of the 
functions output. 

The remainder of this paper is structured as follows: In 
the next section we will introduce some basic definitions 
and notation. In particular we will introduce the concepts of 
Fourier analysis as far as they are relevant to this work. In 



Section III our main results are proven in two steps. First we 
will address Boolean function with uniform distributed input 
variables, secondly the result is extended to the more general 
product distributed case. This is followed by a short discussion 
of our finding (Section [lV) l, before we will conclude with some 
final remarks. 

II. Basic Definitions and Notation 
A. Boolean Functions and Fourier Analysis 

A Boolean function (BE) / e J"™ = {/ : 17" ^ fl} with 
Q, = {—1,+!} maps n-ary input tuples to a binary output. 
Let us consider x = {xi,X2, ■ ■ ■ ,Xn) as an instance of a 
product distributed random vector X = {Xi, X2, ■ ■ ■ , Xn), 
i.e., its probability density functions can be written as 

Px (x) =Pr[X = x] =[]P;^^ (x,). 

i 

Furthermore, let /i^ be the expected value of Xi, i.e., fii = 

E [X.,] and let 



(1) 

be the standard deviation of Xi. It can be easily seen that 

l + ai ■ fii 



X, 



[tti) = 



It is well known that any BE / can be expressed by the 
following sum, called Fourier-expansion, 

/(x) - /('^) ■ -^t^ W' 

UC[n] 

where [n] = {1, 2, . . . , n} and 

M^) = n 



For [/ = we define 00 (x) = 1. The Fourier coefficients 
f{U) can be recovered by 

/([/) = ^ Px(x)-/(x). 0c/ (x). (3) 

X 

If the input variables Xi are uniformly distributed, i.e., fii = 
and (Tj = 1, Eq. Q reduces to 

X(7(x) = 0c/(x) = Yl Xi, 
and consequently, as Px (x) = 2~" for all x, Eq. (jsjl reduces 



to 



/(C/) = 2-"5]/(x).X£/(x). 



B. Canalizing Function 

Definition 1: A BE is called canalizing in variable i, if there 
exists a Boolean restrictive value ai G { — 1,+1} such that the 
function 

/(xU^=J = (4) 

for all xi, ...Xi-i,Xi+i....Xn, where hi G { — 1,+!} is a 
constant. 

As shown in |17| the Eourier coefficients of canalizing 
functions then fulfill the following conditions: 



and 



6, = sgn /(0) , 



(5) 



(6) 



where sgn (•) gives the sign. Hence, a BE is canalizing in i, 
if /(0) and f{{i}) fulfill Eq. (js}. 

III. Mutual Information of Boolean Functions 

The mutual information (MI) between two random variables 
is defined as: 



where 



MI{Y] X) = H{Y) - H{Y\X), 



H{X) = -Y,?x{x)\og^{¥x (x)), 



is Shannon's entropy in bits of some discrete random variable 
X with its domain X. For the special case that \X\ — 2, it 
reduces to the binary entropy function: 

with p — Fx (+!)• Further H{Y\X) is the conditional entropy 
between two discrete random variables X E X and Y E y 



h{y\x)^J2p^ (^)h{y\x 



with 



H{Y\X ^x) = Y, Py\x {y\x) log2 Py\x iy\x) . 
yey 



It has been shown in |16|, that the mutual information 
between one input variable i and the output of a boolean 
function is given as: 



Af/(/(X);X, 
1 



M ^(i + /(0) 



(7) 



E 



hi ^[l + fm+fi{^})h^}(X^) 



The fact that the MI is only depended on /(0) and /({«}) 
coincides with our statement in the previous section, that the 
canalizing property also depend on these two coefficients. 
Hence, we will only focus on those two Fourier coefficients 
in the following considerations. The remaining coefficient can 
be chosen arbitrarily and have no influence on our findings. 
Also, the number of input variables n does not restrict our 
investigations, it only determines the possible values /(0) and 
f({i}), since they are a multiple of 2^". 

A. Mutual Information under Uniform Distribution 

For sake of clarity and ease of comprehension we will first 
focus on canalizing functions in the uniform distributed case. 
In the next section we will then generalize this result to product 
distributed variables. 

Proposition 1: Let / be a Boolean function with uniform 
distributed inputs. For a given and fixed /(0), the mutual 
information (see Eq. (j7|) between one input variable i and 
the output of /, is maximized by all functions, which are 
canalizing in variable i. 
Proof: Since 

of freedom in Eq. (jTl is /({?}). First we show that the mutual 
information is convex with respect to /({i}). Since the first 
summand of Eq. ^ only depends on /(0) we can consider 
it as constant. We hence can focus on the second part, which 
we can write as: 



fJM) is constant, the only remaining degree 

§ is /({?:} 



E 

Xi 



i + ./(0) + /(W)xw(^. 



The binary entropy function h is concave, 
and since its argument is an affine mapping, 
(l + /(0) + /(W)xw(X,))) is convex |18|. 
Finally the expectation is a non-negative weighted sum, 
which preserves convexity, hence the mutual information is 
convex. 

Obviously for /({«}) — 0, the mutual information is 
minimized, hence, due to the convexity, the maximum can be 
found on the boundaries of the domain. The domain is limited 
by the non-negativity of the arguments of h, i.e.. 



< 



1 



i + /(0) + /(«)xw(^O <i- 



Hence, the boundaries are given by 

/(0) + /(«)xw(^.) = ±i- 

It can be seen from Definition [T] that all functions, which 
are canalizing in variable i, are located on the boundary of 



the domain of the mutual information. These function are 
constrained with: 



fi{i}) 



Since ai,bi e { — 1,+1}, there exists four such types of 
functions on the boundary. Looking at their mutual information 
leads us to: 



Af/(/(X);X 
h 



E 



1 + /(0) 



+ /(0) 
h - /(0) 



which yields in: 

M/(/(X);XO 

- Px, {ai) h 



1 + /(0) + (b. - /(0) 



X{t}{Xi) 
X{i}{ai) 



X{i}(o 



=0 



Pjf, {-a^) h 



1 + 



/(0) + {h - /(0) 



and hence: 

Mi{fixy,x,) = h 



l + /(0) 
/(0) + {b 



X{»}(-^a») 



For the uniform distributed case we write: 



A//(/(X);X,) = /i - l + /(0) 



1, /I 



/(0) 



2 V 2 

One can see that bi — sgn (^f{$)j also follows from the this 
equation and due to the symmetry of h, we finally get: 



M/(/(X);XO 



2O 



/(0) 



^Ml/(0)l). 



Hence, the mutual information is independent from a, and hi, 
which concludes the proof. ■ 

B. Mutual Information under Product Distribution 

The result from the previous section can be extended to 
product distributed input variables. We will see, that the 
probability distribution of the canalizing variable plays a key 
role in maximizing the MI. 

Proposition 2: Let / be a Boolean function with product 
distributed inputs. For a given and fixed /(0), the mutual 
information (see Eq. (j7]l) between one input variable i and 
the output of /, is maximized by all functions, which are 
canalizing in variable i, where the canalizing value ai is the 
more probable value, i.e. — sgn(/i,;). 

Proof: The first part of this proof is goes along with the 
proof of Proposition [T] where we simply replace ipu (x) by 




/(0) 



1-1 



/({»}) 



(8) m 



Fig. 1 . Mutual information of BFs with uniform distributed input variables 
versus /(0) and /({«})> one can see that all in i canalizing functions are 
located on the border (black line) 



Starting from Eq. ([8]) we have to analyze four cases, 
namely (a^ = ~IM = -1), (a^ = -1,6^ = +1), (a^ = 
+1, = -1) and (a^ = +1, 6^ = +1). 

We will first show that bi has no influence on the mutual 
information. Assume a; to be either +1 or —1, then using Eq. 
we can write the argument of the second h function in Eq. 
as: 



ff(6.,a,) = ^ (1 + h\fm\ l/(0)l) '^f^^ 

One can easily see, that g{+\,ai) = 1 — g(— l,aj) and due 
to the symmetry of the binary entropy, bi has no influence on 
the /i-function and hence on the MI. 

Let us now investigate the dependence of ai on g. We set 
w.l.o.g. bi = +1. Using Eqs. (j2| and ([T]) we get: 

a.) + 1/(0)1 + (1 - 1/(0)1^ ^ "'^'^ 

which is clearly larger than ^ and hence must be maximized to 
maximize the MI. Since ^1 — |/(0)|^ > 0, we need to make 



large. This is the case if has the same sign as its 
expectation value /i^, i.e. a; — sgn(/Xi). This also minimizes 
flj), which concludes the proof. ■ 

IV. Discussion and Conclusion 

To visualize our findings we plotted in Figure [T| a 3D 
diagram of the mutual information of a BE with uniform 
distributed input variables versus /(0) and f{{i}). In Figure|2] 
we present a projection of the surface in the (/(0), i\f /)-plane. 
It can be seen from these pictures that the canalizing function 
form the boundary of the domain of the MI. Further the 
symmetry with respect to and bi — sgn (/(0)) can be seen. 
In addition it becomes visible, that the mutual information also 
depends of the actual zero coefficient. This is mainly due to 
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Fig. 2. Mutual information of Fig.[T]projected in tlie (/(0), M/)-plane, one 
can see tiiat all in i canalizing functions are located on the border (black line) 




Fig. 4. Mutual information of Fig.jsjprojected in the (/(0), M/)-plane, one 
can see that all in i canalizing functions are located on the border (black line) 




1-1 /(W) 



Fig. 3. Mutual information of a BF with product distributed input variables 
versus /(0) and /({*}), Pi = 0.3, one can see that all in i canalizing 
functions are located on the border (black line) 



the first term of the MI (Eq. |7]), the entropy of the functions 
output. 

In Figures [3] and [4] the same plots can be found for product 
distributed input variables, with pi = 0.3. Here, the skew of 
the mutual information towards the more probable canalizing 
value and the symmetry with respect to bi can bee seen. 

Our findings show the optimality of canalizing functions 
with respect to information processing abilities. Further it has 
been stated in literature |8|, that canalizing functions have a 
stabilizing influence on the network dynamics. This supports 
the conjectures 13], H, ||5|, that these two properties are 
closely related. 

An open problem remains the impact of canalizing function 
on the mutual information between a set of variables and the 
function's output. One may presume that based on the results 
of this paper, that it is maximized by functions, which are 



somehow canalizing in all that variables. 
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